cc @edersondisouza - do you know if there is a way to implement the missing functions for those few platforms?

@dcpleung can you please take a look.

@nashif - I think I've found a better way. Will update this PR shortly. I do not have an intel_ish_5_4_1 device (afaik). Can you or @dcpleung please have a run with it?

Testing done:

# qemu_x86 still works, as expected
west build -p -b qemu_x86 -t run tests/posix/xsi_realtime
...
SUITE PASS - 100.00% [xsi_realtime]: pass = 11, fail = 0, skip = 0, total = 11 duration = 0.131 seconds
 - PASS - [xsi_realtime.test_fs_datasync] duration = 0.002 seconds
 - PASS - [xsi_realtime.test_fs_sync] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue] duration = 0.013 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_basic] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_errors] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_non_empty_queue] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_thread] duration = 0.106 seconds
 - PASS - [xsi_realtime.test_shm_mmap] duration = 0.003 seconds
 - PASS - [xsi_realtime.test_shm_open] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_shm_read_write] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_shm_unlink] duration = 0.001 seconds

# qemu_x86_64 still works, as expected
west build -p -b qemu_x86_64 -t run tests/posix/xsi_realtime
...
SUITE PASS - 100.00% [xsi_realtime]: pass = 11, fail = 0, skip = 0, total = 11 duration = 0.147 seconds
 - PASS - [xsi_realtime.test_fs_datasync] duration = 0.004 seconds
 - PASS - [xsi_realtime.test_fs_sync] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue] duration = 0.025 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_basic] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_errors] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_non_empty_queue] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_mqueue_notify_thread] duration = 0.110 seconds
 - PASS - [xsi_realtime.test_shm_mmap] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_shm_open] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_shm_read_write] duration = 0.001 seconds
 - PASS - [xsi_realtime.test_shm_unlink] duration = 0.001 seconds
 
# intel_ish_5_4_1, links (and should run with the mmap tests skipped)
west build -p auto -b intel_ish_5_4_1 tests/posix/common
...
Generating files from /Users/cfriedt/zephyrproject/zephyr/build/zephyr/zephyr.elf for board: intel_ish_5_4_1
    Packing EC image file for ISH
      kernel binary size: 94972

# intel_ish_541, links and should run as expected
west build -p auto -b intel_ish_5_4_1 tests/lib/c_lib/thrd
...
Generating files from /Users/cfriedt/zephyrproject/zephyr/build/zephyr/zephyr.elf for board: intel_ish_5_4_1
    Packing EC image file for ISH
      kernel binary size: 64380

Hm?... why is CONFIG_MMU enabled without CONFIG_X86_MMU? CONFIG_X86_MMU selects CONFIG_MMU and not the other way around. Another example is that CONFIG_ARM_MPU selects CONFIG_MPU. Maybe the option enabling CONFIG_MMU should instead be depending on it?

@cfriedt should posix be implying/selecting MMU in the first place? this is something that needs to be set by the platform, so something like this:

config POSIX_MAPPED_FILES
        bool "POSIX memory-mapped files"
        # disable Xtensa for now until linker issues are resolved
        imply MMU if (CPU_HAS_MMU && !XTENSA)

should depend on MMU and not imply/select it

Imply / select is correct here.

The reason being that posix option groups are effectively shorthands for manually specifying a lot of dependencies.

Imply is needed here because some architectures / platforms simply cannot support mmap.

I still think it is backwards. CONFIG_POSIX_MAPPED_FILES should depend on MMU (and default to y if MMU), instead of enabling MMU (in a sense forcibly). What if there is a board using a MMU capable CPU but the application does not need MMU?

I still think it is backwards. CONFIG_POSIX_MAPPED_FILES should depend on MMU (and default to y if MMU), instead of enabling MMU (in a sense forcibly). What if there is a board using a MMU capable CPU but the application does not need MMU?

CONFIG_POSIX_MAPPED_FILES should not default to y if MMU is selected. It should default to n (in general), which is how Zephyr has always worked.

The nature of POSIX Option Groups, Options, and Subprofiles is that they should be selected. All POSIX Subprofiles operate by selecting a predefined subset of Subprofiling Option Groups. In that sense, POSIX subprofiles operate as a shorthand.

CONFIG_POSIX_MAPPED_FILES is selected by generally every possible POSIX subprofile since it's required by CONFIG_POSIX_AEP_CHOICE_BASE (i.e. the mandatory system interfaces). Naturally, some architectures and platforms cannot support mapped memory, hence the imply.

There are a couple of issues still in need of attention which will alleviate concerns

• separate C11 threads from POSIX (aka, make C11 threads use a native Zephyr thread pool)
  • CONFIG_POSIX_AEP_CHOICE_BASE not enabled (which solves the issues above)
  • CONFIG_POSIX_AEP_CHOICE_BASE then becomes roughly the perceptual equivalent of CONFIG_POSIX_API
• POSIX kconfigs show up although POSIX APIs is not being used #75843, which has a planned fix, roughly outlined below
  • once CONFIG_POSIX_AEP_CHOICE_BASE is not selected by C11 threads, the problems above no longer exist, and we can simplify configuration (and subsequently active POSIX options)
  • probably should enable things like CONFIG_POSIX_C_LANG_SUPPORT_R or CONFIG_POSIX_C_LIB_EXT which Zephyr seems to want by default anyway (e.g. strtok_r(), gmtime_r(), strnlen())
  • the above options can be independently enabled without enabling CONFIG_POSIX_AEP_CHOICE_BASE, since they do not rely on any POSIX functionality

Realistically, if we want a consistent arch interface, which is the underlying reason for the regression (not really anything to do with POSIX), someone should implement even a trivial or non-functional version of arch_mem_map() and arch_mem_unmap() for the outlying x86 platforms that do not provide it.

That should naturally be something that is done by an x86 arch maintainer.

Hm?... why is CONFIG_MMU enabled without CONFIG_X86_MMU? CONFIG_X86_MMU selects CONFIG_MMU and not the other way around. Another example is that CONFIG_ARM_MPU selects CONFIG_MPU. Maybe the option enabling CONFIG_MMU should instead be depending on it?

Right.

same thing for MPU.

Nothing else should be enabling MMU, neither by selection nor by implying.

I still think it is backwards. CONFIG_POSIX_MAPPED_FILES should depend on MMU (and default to y if MMU), instead of enabling MMU (in a sense forcibly). What if there is a board using a MMU capable CPU but the application does not need MMU?

CONFIG_POSIX_MAPPED_FILES should not default to y if MMU is selected. It should default to n (in general), which is how Zephyr has always worked.

The nature of POSIX Option Groups, Options, and Subprofiles is that they should be selected. All POSIX Subprofiles operate by selecting a predefined subset of Subprofiling Option Groups. In that sense, POSIX subprofiles operate as a shorthand.

CONFIG_POSIX_MAPPED_FILES is selected by generally every possible POSIX subprofile since it's required by CONFIG_POSIX_AEP_CHOICE_BASE (i.e. the mandatory system interfaces). Naturally, some architectures and platforms cannot support mapped memory, hence the imply.

There are a couple of issues still in need of attention which will alleviate concerns

• separate C11 threads from POSIX (aka, make C11 threads use a native Zephyr thread pool)

  • CONFIG_POSIX_AEP_CHOICE_BASE not enabled (which solves the issues above)
  • CONFIG_POSIX_AEP_CHOICE_BASE then becomes roughly the perceptual equivalent of CONFIG_POSIX_API

• POSIX kconfigs show up although POSIX APIs is not being used #75843, which has a planned fix, roughly outlined below

  • once CONFIG_POSIX_AEP_CHOICE_BASE is not selected by C11 threads, the problems above no longer exist, and we can simplify configuration (and subsequently active POSIX options)
  • probably should enable things like CONFIG_POSIX_C_LANG_SUPPORT_R or CONFIG_POSIX_C_LIB_EXT which Zephyr seems to want by default anyway (e.g. strtok_r(), gmtime_r(), strnlen())
  • the above options can be independently enabled without enabling CONFIG_POSIX_AEP_CHOICE_BASE, since they do not rely on any POSIX functionality

I do not see how any of this is related.

Realistically, if we want a consistent arch interface, which is the underlying reason for the regression (not really anything to do with POSIX), someone should implement even a trivial or non-functional version of arch_mem_map() and arch_mem_unmap() for the outlying x86 platforms that do not provide it.

That should naturally be something that is done by an x86 arch maintainer.

The reality is, MMU was enabled first on X86 and X86 has been the leading architecture for supporting MMU and everything related to MMU, so there is nothing missing and nothing to be implemented. If MMU is enabled (selected) via X86_MMU, then all of those interfaces are available, however, if you select MMU directly in a higher layer and if the platform does not do that for some reason, then things fail.

We are looking to see how to avoid this and resolve this with this particular platform, however, selecting MMU directly is not correct per the current design, see comment by @dcpleung , the maintainer of MMU in Zephyr.

I do not see how any of this is related.

Maybe you should try to

if you select MMU directly in a higher layer

Imply, when the application enables the feature that requires an mmu, but that was already mentioned, iirc.

and if the platform does not do that for some reason, then things fail.

Hence the workaround. "Some reason", in this case is that MMU support for x86 is somewhat inconsistent.

We are looking to see how to avoid this and resolve this with this particular platform,

Same!

however, selecting MMU directly is not correct per the current design

Implying, if the hardware supports it, and actually, it is correct.

In libraries.libc.c11_threads, POSIX_MAPPED_FILES is enabled (likely due to C11 threads pulling in generic POSIX features, which would ideally not be the case), and MMU support is implied if the hardware supports it (since that is a hardware feature required for POSIX_MAPPED_FILES). It is not enabled on architectures without MMU support.

Ideally, we could have C11 thread support without POSIX. I'll see if I can create a separate PR for that, which would prevent that failure in that testsuite (specifically by not enabling MMU when it is not needed).

However, the other POSIX tests that are failing legitimately have POSIX_MAPPED_FILES enabled because that needed by the spec (unless hardware does not support it). So the failure mode is legitimate and a workaround is needed for x86.

This PR simply adds missing dependencies for x86 in that the x86 MMU must be available on the x86 architecture for MMU to be enabled.

Seems pretty logical to me.

if you select MMU directly in a higher layer

Imply, when the application enables the feature that requires an mmu, but that was already mentioned, iirc.

you should neither imply nor select MMU, this is done by the arch specific implementation of MMU. Please stop repeating what was already said multiple times already. MMU shall not be enabled by a software layer, this is a hardware feature that is enabled by the platform .

and if the platform does not do that for some reason, then things fail.

Hence the workaround. "Some reason", in this case is that MMU support for x86 is somewhat inconsistent.

"Some reason" is the fact you can have MMU supported by hardware but you have the choice of not enabling it, this is as simple as that. This has nothing with X86 being inconsistent. I can do the same on ARM and Xtensa for MMUs and the same for MPUs.

however, selecting MMU directly is not correct per the current design

Implying, if the hardware supports it, and actually, it is correct.

That is not something you do at this level, enabling hardware features happens at the platform/arch level. You should not be assuming the platform wants or is able to enable a certain hardware feature because it is supported by the hardware. A platform can selectively enable hardware features, if they are advertised as supported.

In libraries.libc.c11_threads, POSIX_MAPPED_FILES is enabled (likely due to C11 threads pulling in generic POSIX features, which would ideally not be the case), and MMU support is implied if the hardware supports it (since that is a hardware feature required for POSIX_MAPPED_FILES). It is not enabled on architectures without MMU support.

yeah, the thing is, some platforms of arch XYZ will not enable all features supported by arch XYZ, so you should not assume that a feature is supported by the platform if the architecture advertises that as supported.

However, the other POSIX tests that are failing legitimately have POSIX_MAPPED_FILES enabled because that needed by the spec (unless hardware does not support it). So the failure mode is legitimate and a workaround is needed for x86.

No, no workarounds are needed for X86. The issue is not related to X86.

I have shown in the linked bug this can happen on ARM as well

This PR simply adds missing dependencies for x86 in that the x86 MMU must be available on the x86 architecture for MMU to be enabled.

Seems pretty logical to me.

There is nothing logical about this. This is uneeded workaround. Posix should not enable MMU, it should depend on it.

If MMU is only meant to be enabled by the architecture, then why is this a user-configurable option?

Similarly, why do we have e.g. CONFIG_I2C or any hardware or architecture level Kconfig options that users can choose to enable or disable?

Why is it that an application can choose to enable MMU in one case but cannot choose to enable MMU in another case?

If an application chooses to enable the standard POSIX system interfaces, then they typically want the features that would provide them.

The argument against seems quite inconsistent. I'll try to find the arm (aarch64?) bug that was referenced as well...

If MMU is only meant to be enabled by the architecture, then why is this a user-configurable option?

Even if we make it promptless, the option would still be selectable. There were only 3 ocurrances in the tree with MMU being set in project configs and they were wrong/not needed, probably leftovers from debugging... see #85834

MMU could be made promptless, that could be done as an improvement.

Similarly, why do we have e.g. CONFIG_I2C or any hardware or architecture level Kconfig options that users can choose to enable or disable?

if you select/imply I2C r set CONFIG_I2C in your prj.conf without a supporting driver or implementation, you end up in the same situation as with MMU, no?

i2c driver instances in the tree 'select I2C' etc. It is in many ways similar.

Why is it that an application can choose to enable MMU in one case but cannot choose to enable MMU in another case?

Where do you see applications enabling MMU?

If an application chooses to enable the standard POSIX system interfaces, then they typically want the features that would provide them.

You will need to depend on those features. POSIX should not change how a plaform is setup. If you enable MMU on a platform that was not setup for MMU, things break.

The argument against seems quite inconsistent. I'll try to find the arm (aarch64?) bug that was referenced as well...

It is actually very consistent, nobody in the tree enables MMU config option outside of platforms/arch (and now POSIX).
And as was said initially by @dcpleung

Hm?... why is CONFIG_MMU enabled without CONFIG_X86_MMU? CONFIG_X86_MMU selects CONFIG_MMU and not the other way around. Another example is that CONFIG_ARM_MPU selects CONFIG_MPU. Maybe the option enabling CONFIG_MMU should instead be depending on it?

This is the current state of things, this applies to everything in the tree, not only X86. Enabling MMU without enabling the MMU implementation is not supposed to work. Maybe we need to document this in the MMU kconfig to make it clear.

Closing in favour of #85837